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Diagrammatic  reasoning  is  ubiquitous  in  Army  reasoning:  situation  understanding  and 
planning  in  the  Army  both  involve  representing  aspects  of  the  situation  and  plans  in  the  form  of 
diagrams.  We  have  been  developing  a  general  architecture  to  support  diagrammatic  reasoning 
for  Army  applications,  and  in  an  earlier  report  [1]  we  discussed  an  application  in  simple 
maneuver  recognition.  Our  research  strategy  has  been  to  investigate  a  variety  of  applications, 
each  bringing  additional  requirements  for  perception  and  diagrammatic  object  creating 
capabilities  that  can  assist  the  army.  One  such  area  is  the  army’s  All  Source  Analysis  System 
(ASAS)  that  is  designed  to  automate  the  processing  and  analysis  of  intelligence  data  from  all 
possible  sources.  One  of  its  purposes  is  to  keep  track  of  various  enemy  assets,  based  on  a  variety 
of  sensors  and  reports,  including  direct  sightings  of  entities,  hi  order  to  maintain  a  coherent 
view,  the  system  has  to  decide  whether  a  new  sighting  refers  to  a  new  entity,  a  previously  known 
entity  that  has  since  moved  (re-identification)  or  is  erroneous.  This  is  the  entity  re-identification 
task.  We  have  a  built  a  system  that  uses  an  abductive  reasoning  process  together  with  a 
diagrammatic  reasoning  system  to  solve  this  problem.  In  this  paper,  we  look  at  some  of  the 
issues  that  the  entity  re-identification  task  poses  for  diagrammatic  reasoning. 

When  information  in  a  set  of  premises  is  represented  in  a  diagram,  information  that 
follows  from  the  premises  is  often  available  in  the  same  diagram  to  be  picked  up  by  perception, 
and  often  this  additional  information  is  called  a  free  ride.  A  physical  diagram  (on  a  screen  or  on 
paper)  is  an  image  that  contains  diagrammatic  objects  -  points,  curves  and  regions  -  that 
represent  objects  of  interest  in  the  domain  of  discourse.  Our  diagram  representation  structure 
(DRS)  is  a  data  type  that  contains  representations  of  the  diagrammatic  objects  in  the  image.  The 
objects  in  DRS  have  associated  with  them  information  about  their  spatiality  —  locations  for  point 
objects,  sets  of  points  that  constitute  the  objects  for  curves  and  regions.  The  DRS  architecture  to 
be  discussed  in  this  paper  is  an  improved  version  of  the  one  in  [1], 

There  are  two  types  of  perception  routines:  the  ones  in  the  first  type  re-perform  the 
figure-ground  separation  on  the  image  -  rather  than  on  the  DRS  -  perceiving  what  we  call 
emergent  objects  (e.g.,  the  two  sub-regions  that  emerge  when  a  curve  intersects  a  region.) 
Routines  of  the  second  type  perceive  spatial  properties  of  objects  and  spatial  relations  between 
objects.  These  routines  work  from  DRS.  DRS  thus  is  an  intermediate  representation  that 
supports  reconstituting  the  image,  needed  for  emergent  object  identification,  and  also  the 
perceptual  routines  that  perceive  properties  of  and  relations  between  objects.  More  information 
on  the  Diagrammatic  Representation  System  and  the  accompanying  perceptual  routines  can  be 
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found  in  references  [1]  and  [2],  Another  set  of  routines,  called  action  routines,  create 
diagrammatic  objects  that  satisfy  specific  perceptual  criteria,  such  as  “a  curve  object  that 
intersects  a  given  region  object,”  and  “a  point  object  inside  the  region  object.” 

The  diagram  on  the  left  can  be  used  to  illustrate  certain  subtasks 
that  arise  in  an  entity  re-identification  problem,  and  that  are  especially 
appropriate  for  diagrammatic  reasoning.  In  order  to  decide  if  a  newly 
sighted  object  in  an  entity  in  the  data  base,  the  system  needs  to  come  up 
with  a  set  of  possible  paths  from  the  entity’s  previously  known  location 
to  the  location  of  the  new  sighting.  T1  and  T2  are  two  tanks  that  are  in 
the  system.  T3  is  a  tank  that  has  been  sighted  at  a  later  time.  In  order  for 
T3  to  be  T1  (T2),  at  least  one  path  should  exist  from  T1  (T2)  to  T3.  Path  finding  is  complicated 
by  the  fact  that  the  area  of  operations  is  over  open  terrain  where  the  number  of  possible  routes  is 
often  infinite.  Thus,  any  path  finding  algorithm  must  be  able  to  collapse  these  infinite  number  of 
paths  into  a  set  of  paths  where  each  member  of  the  set  is  a  representative  of  a  class  of  paths. 
These  paths,  called  representative  paths,  can  then  be  subjected  to  further  scrutiny,  such  as 
ensuring  that  the  time  taken  to  traverse  the  path  is  less  than  the  time  between  reports  and,  if 
needed,  modifying  them  to  satisfy  various  spatial  and  temporal  constraints.  Another  subtask 
involves  detecting  whether  a  path  intersects  a  sensor  field.  If  a  path  is  found  to  intersect  a  sensor 
field,  as  in  the  figure  above,  and  there  is  no  corresponding  sensor  activity,  the  system  has  to  tiy 
and  modify  this  path  to  see  if  it  can  avoid  the  sensor  field.  The  path  that  is  so  modified  has  the 
added  constraint  that  it  should  remain  within  the  same  representative  class.  An  interesting 
corollary  to  this  is  when  there  is  a  sensor  report  of  activity,  but  the  path  generated  by  the  system 
does  not  go  over  the  sensor  area.  In  this  case,  the  system  has  to  try  and  modify  the  path  such  that 
it  does  intersect  with  the  sensor  region  while  still  satisfying  the  various  constraints  such  as  time 
to  traverse  and  retaining  its  membership  in  the  same  representative  class.  There  are  two 
companion  paper  abstracts  that  are  also  being  submitted  for  presentation  at  the  conference:  one 
on  the  fusion  reasoning  that  uses  abductive  inference  to  solve  the  overall  re-identification  task 
[3],  and  the  other  [4]  on  the  computational  complexity  of  the  large  suite  of  perceptual  and  action 
routines  that  we  are  building  for  diagrammatic  reasoning  in  general.  In  contrast  to  [4],  the  full 
paper  will  focus  on  the  routines  that  are  especially  required  for  the  entity  re- identification  task, 
how  the  task  is  represented  in  DRS,  and  the  general  DRS/Image  architecture. 
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